GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators

Imitation learning from human demonstrations is a powerful framework to teach robots new skills. However, the performance of the learned policies is bottlenecked by the quality, scale, and variety of the demonstration data. In this paper, we aim to lower the barrier to collecting large and high-quality human demonstration data by proposing GELLO, a general framework for building low-cost and intuitive teleoperation systems for robotic manipulation. Given a target robot arm, we build a GELLO controller that has the same kinematic structure as the target arm, leveraging 3D-printed parts and off-the-shelf motors. GELLO is easy to build and intuitive to use. Through an extensive user study, we show that GELLO enables more reliable and efficient demonstration collection compared to commonly used teleoperation devices in the imitation learning literature such as VR controllers and 3D spacemouses. We further demonstrate the capabilities of GELLO for performing complex bi-manual and contact-rich manipulation tasks. To make GELLO accessible to everyone, we have designed and built GELLO systems for 3 commonly used robotic arms: Franka, UR5, and xArm. All software and hardware are open-sourced and can be found on our website: https://wuphilipp.github.io/gello/

Robust surface segmentation and edge feature lines extraction from fractured fragments of relics

AbstractSurface segmentation and edge feature lines extraction from fractured fragments of relics are essential steps for computer assisted restoration of fragmented relics. As these fragments were heavily eroded, it is a challenging work to segment surface and extract edge feature lines. This paper presents a novel method to segment surface and extract edge feature lines from triangular meshes of irregular fractured fragments. Firstly, a rough surface segmentation is accomplished by using a clustering algorithm based on the vertex normal vector. Secondly, in order to differentiate between original and fracture faces, a novel integral invariant is introduced to compute the surface roughness. Thirdly, an accurate surface segmentation is implemented by merging faces based on face normal vector and roughness. Finally, edge feature lines are extracted based on the surface segmentation. Some experiments are made and analyzed, and the results show that our method can achieve surface segmentation and edge extraction effectively

A novel statistical cerebrovascular segmentation algorithm with particle swarm optimization

AbstractWe present an automatic statistical intensity-based approach to extract the 3D cerebrovascular structure from time-of flight (TOF) magnetic resonance angiography (MRA) data. We use the finite mixture model (FMM) to fit the intensity histogram of the brain image sequence, where the cerebral vascular structure is modeled by a Gaussian distribution function and the other low intensity tissues are modeled by Gaussian and Rayleigh distribution functions. To estimate the parameters of the FMM, we propose an improved particle swarm optimization (PSO) algorithm, which has a disturbing term in speeding updating the formula of PSO to ensure its convergence. We also use the ring shape topology of the particles neighborhood to improve the performance of the algorithm. Computational results on 34 test data show that the proposed method provides accurate segmentation, especially for those blood vessels of small sizes

Multiattention Adaptation Network for Motor Imagery Recognition

This work was supported in part by the National Natural Science Foundation of China under Grants Nos. 61873181 and 61922062Peer reviewedPostprin

A Dense Point-to-Point Alignment Method for Realistic 3D Face Morphing and Animation

We present a new point matching method to overcome the dense point-to-point alignment of scanned 3D faces. Instead of using the rigid spatial transformation in the traditional iterative closest point (ICP) algorithm, we adopt the thin plate spline (TPS) transformation to model the deformation of different 3D faces. Because TPS is a non-rigid transformation with good smooth property, it is suitable for formulating the complex variety of human facial morphology. A closest point searching algorithm is proposed to keep one-to-one mapping, and to get good efficiency the point matching method is accelerated by a KD-tree method. Having constructed the dense point-to-point correspondence of 3D faces, we create 3D face morphing and animation by key-frames interpolation and obtain realistic results. Comparing with ICP algorithm and the optical flow method, the presented point matching method can achieve good matching accuracy and stability. The experiment results have shown that our method is efficient for dense point objects registration

A Smoothed Finite Element-Based Elasticity Model for Soft Bodies

One of the major challenges in mesh-based deformation simulation in computer graphics is to deal with mesh distortion. In this paper, we present a novel mesh-insensitive and softer method for simulating deformable solid bodies under the assumptions of linear elastic mechanics. A face-based strain smoothing method is adopted to alleviate mesh distortion instead of the traditional spatial adaptive smoothing method. Then, we propose a way to combine the strain smoothing method and the corotational method. With this approach, the amplitude and frequency of transient displacements are slightly affected by the distorted mesh. Realistic simulation results are generated under large rotation using a linear elasticity model without adding significant complexity or computational cost to the standard corotational FEM. Meanwhile, softening effect is a by-product of our method

3D Facial Similarity Measure Based on Geodesic Network and Curvatures

Automated 3D facial similarity measure is a challenging and valuable research topic in anthropology and computer graphics. It is widely used in various fields, such as criminal investigation, kinship confirmation, and face recognition. This paper proposes a 3D facial similarity measure method based on a combination of geodesic and curvature features. Firstly, a geodesic network is generated for each face with geodesics and iso-geodesics determined and these network points are adopted as the correspondence across face models. Then, four metrics associated with curvatures, that is, the mean curvature, Gaussian curvature, shape index, and curvedness, are computed for each network point by using a weighted average of its neighborhood points. Finally, correlation coefficients according to these metrics are computed, respectively, as the similarity measures between two 3D face models. Experiments of different persons’ 3D facial models and different 3D facial models of the same person are implemented and compared with a subjective face similarity study. The results show that the geodesic network plays an important role in 3D facial similarity measure. The similarity measure defined by shape index is consistent with human’s subjective evaluation basically, and it can measure the 3D face similarity more objectively than the other indices

Modeling and Rendering with eXpressive B-Spline Curves

This research is supported by the Ministry of Education, Singapore, under its Academic Research Fund Tier 1 (RG 22/20). Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not reflect the views of the Ministry of Education, Singapore.eXpressive B-Spline Curve (XBSC) is a resolution-independent and computationally efficient technique for vector-based stroke modeling and rendering with the flexibility in defining and adjusting the shape and other parameters of the stroke. It generalizes the existing Disk B-Spline Curve (DBSC) geometric representation, which itself is a generalization of the Disk Bézier curve. XBSC allows flexible shape and color manipulation and rendering of strokes with asymmetrical shape control and rich color management. These properties make XBSC suitable for modeling freeform stroke shapes and animation, specifically in squash and stretch, a common technique to bestow elasticity and flexibility in shape changes. During the squash and stretch animation computation, we constrain the shape of the XBSC stroke to conserve its area. To achieve this, we apply the simulated annealing algorithm to iteratively adjust the XBSC while maintaining its area. We show several drawings, rendering and deformation examples to demonstrate the robustness of XBSC

Geodesic stripes based hierarchical evaluation for 3D facial similarity

Similarity evaluation of 3D face is the core issue in 3D face recognition. The article puts forwards a geodesic stripes based evaluation method which realizes assessment from global face to local parts. It can be used to make effective and comprehensive evaluation in many fields such as 3D face reconstruction, forensic science, archaeology etc. First, simplify each 3D face with a series of geodesic stripes and calculate distribution vector between each pair of stripes, which reflects 3D space distribution feature. Then through feature extraction on entire face, we get a distribution matrix which consists of all distribution vectors. The similarity between two distribution matrices directly shows the global similarity between two faces. We also extract geodesic stripes feature on local organs like mouth eye and nose to make a more accurate evaluation. The experimental results on SHREC2008 3D face database further testify that the hierarchical evaluation method is available and consistent with the subjective evaluation